Driving assistance apparatus and image processing method

ABSTRACT

Disclosed is a driving assistance apparatus which can be mounted on a vehicle. The driving assistance apparatus includes an image capturing part, and a processor configured to identify a specific color area in a peripheral image captured by the image capturing part as a first candidate area, identify an area including a specific shape in the first candidate area as a second candidate area, and identify an area of a traffic sign in the second candidate area.

BACKGROUND Field

The disclosure relates to a driving assistance apparatus and an imageprocessing method. More particularly, the disclosure relates to adriving assistance apparatus for identifying a traffic sign and an imageprocessing method.

Description of Related Art

Traffic signs promote traffic safety and make traffic flow smoothly in aconsistent and unified way to road users, and provide variousinformation required to protect road facilities. Since these trafficsigns can exist anywhere on the road, an entire area of an imagecaptured in real time must be searched for effective recognition.

However, in the related art, excessive calculations were consumed forsearching the entire image area, which made real-time search difficult,or the amount of calculation was reduced, thereby decreasing arecognition rate.

SUMMARY

The disclosure is to provide a driving assistance apparatus thatidentifies a candidate area of traffic signs based on color informationand shape information of a captured image to increase identificationperformance of traffic signs and to reduce the amount of calculation,and an image processing method.

In addition, an objective of the disclosure is to make traffic signsdetected robustly against damage or contamination of the sign indetecting a traffic sign area.

According to an embodiment of the disclosure, a driving assistanceapparatus includes an image capturing part, and a processor configuredto identify a specific color area in a peripheral image captured by theimage capturing part as a first candidate area, identify an areaincluding a specific shape in the first candidate area as a secondcandidate area, and identify an area of a traffic sign in the secondcandidate area.

The processor may be configured to identify the second candidate area asa plurality of first blocks based on a first direction, and acquirefirst pixel information based on a pixel value included in each of theplurality of first blocks, identify the second candidate area as aplurality of second blocks based on a second direction, and acquiresecond pixel information based on a pixel value included in each of theplurality of second blocks, and identify an area of the traffic signbased on the first pixel information and the second pixel information.

The first pixel information may be configured to include a differencevalue between an average value of pixels included in each of theplurality of first blocks, and an average value of pixels included inadjacent first blocks, and wherein the second pixel information isconfigured to include a difference value between an average value ofpixels included in each of the plurality of second blocks, and anaverage value of pixels included in adjacent second blocks.

The processor may be configured to apply the first pixel information andthe second pixel information to a training model to identify the area ofthe traffic sign.

The training model may be configured to identify a plurality of sampletraffic sign images as a plurality of blocks, and learn and obtain pixelinformation based on pixel values included in each of the plurality ofblocks.

The processor may be configured to identify red (R) color, green (G)color, and blue (B) color included in the peripheral image, and identifyan area in which a pixel value of the R color is equal to or greaterthan a predetermined multiple of pixel values of the G color and the Bcolor as the first candidate area.

The apparatus may further include at least one among illuminance sensorand rain sensor.

The pre-determined multiple may be configured to be updated based on atleast one of illuminance measured by the illuminance sensor and rainfallmeasured by the rain sensor.

The processor may be configured to, based on the first candidate areabeing identified, include a binary image including only the firstcandidate area among the peripheral images, and identify the secondcandidate area based on the acquired binary image.

The apparats may further include a storage configured to store referenceshape information of a traffic sign, based on the reference shapeinformation being identified from the first candidate area, andidentifying the first candidate area as the second candidate area.

The apparatus may further include a display.

The processor may be configured to control the display to identify andprovide at least one of a type and an instruction content of the trafficsign in the identified traffic sign area, and based on at least one ofthe type and instruction content of the traffic sign, control thevehicle.

According to an embodiment of the disclosure, a method of imageprocessing includes acquiring a peripheral image of a vehicle through animage capturing part, identifying a first candidate area based on colorinformation included in the acquired peripheral image, identifying asecond candidate area based on shape information included in the firstcandidate area, and identifying the second candidate area as a pluralityof blocks, and identifying an area of traffic sign based on pixel valuesincluded in each of the plurality of blocks.

The identifying the traffic sign area may be configured to identify thesecond candidate area as a plurality of first blocks based on a firstdirection, and acquire first pixel information based on a pixel valueincluded in each of the plurality of first blocks, identify the secondcandidate area as a plurality of second blocks based on a seconddirection, and acquire second pixel information based on a pixel valueincluded in each of the plurality of second blocks, and identify an areaof the traffic sign based on the first pixel information and the secondpixel information.

The first pixel information may be configured to include a differencevalue between an average value of pixels included in each of theplurality of first blocks, and an average value of pixels included inadjacent first blocks.

The second pixel information may be configured to include a differencevalue between an average value of pixels included in each of theplurality of second blocks, and an average value of pixels included inadjacent second blocks.

The identifying the traffic sign area may be configured to apply thefirst pixel information and the second pixel information to a trainingmodel to identify the area of the traffic sign.

The training model may be configured to identify a plurality of sampletraffic sign images as a plurality of blocks, and learn and obtain pixelinformation based on pixel values included in each of the plurality ofblocks.

The identifying the first candidate area may be configured to identifyred (R) color, green (G) color, and blue (B) color included in theperipheral image, and identify an area in which a pixel value of the Rcolor is equal to or greater than a predetermined multiple of pixelvalues of the G color and the B color as the first candidate area.

The predetermined multiple may be configured to be updated based on atleast one of illuminance and rainfall.

The identifying the second candidate area may include, based on thefirst candidate area being identified, acquiring a binary imageincluding only the first candidate area among the peripheral images, andidentifying the second candidate area based on the acquired binaryimage.

The identifying the second candidate area may include, based on thereference shape information of a traffic sign pre-stored in the firstcandidate area being identified in the first candidate area, identifyingthe first candidate area as the second candidate area.

The image processing method may further include providing at least oneof a type and an instruction content of the traffic sign in theidentified traffic sign area, and based on at least one of the type andinstruction content of the traffic sign, controlling the vehicle.

As described above, according to various embodiments of the disclosure,since the driving assistance apparatus is based on color information andshape information of a captured image, it is possible to increaseaccuracy of identifying a candidate area of a traffic sign.

In addition, since the apparatus is based on pixel value informationincluded in the captured image, the amount of computation consumed toidentify a traffic sign is reduced, and traffic sign information isprovided to a user in real time, thereby helping safe driving.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view illustrating a peripheral image including a trafficsign to support understanding of the disclosure;

FIG. 2 is a block diagram indicating a configuration of a drivingassistance apparatus according to an embodiment of the disclosure;

FIG. 3 is a block diagram illustrating an example of a detailedconfiguration of a driving assistance apparatus of FIG. 2 ;

FIGS. 4A and 4B are views illustrating a process of identifying a firstcandidate area according to an embodiment of the disclosure;

FIGS. 5A and 5B are views illustrating a process of identifying a secondcandidate area according to an embodiment of the disclosure;

FIGS. 6A and 6B are views illustrating a process of acquiring pixelinformation by identifying a second candidate area as a plurality ofblocks according to an embodiment of the disclosure;

FIGS. 7A and 7B are views illustrating an operation of providinginformation on an identified traffic sign area to a user according to anembodiment of the disclosure; and

FIG. 8 is a flowchart illustrating a method of controlling a drivingassistance apparatus according to an embodiment of the disclosure.

DETAILED DESCRIPTION

Hereinafter, the disclosure will be described in detail with referenceto the accompanying drawings.

The terms used in example embodiments will be briefly explained, andexample embodiments will be described in greater detail with referenceto the accompanying drawings.

Terms used in the disclosure are selected as general terminologiescurrently widely used in consideration of the configuration andfunctions of the disclosure, but can be different depending on intentionof those skilled in the art, a precedent, appearance of newtechnologies, and the like. Further, in specific cases, terms may bearbitrarily selected. In this case, the meaning of the terms will bedescribed in the description of the corresponding embodiments.Accordingly, the terms used in the description should not necessarily beconstrued as simple names of the terms, but be defined based on meaningsof the terms and overall contents of the disclosure.

The example embodiments may vary, and may be provided in differentexample embodiments. Various example embodiments will be described withreference to accompanying drawings. However, this does not necessarilylimit the scope of the exemplary embodiments to a specific embodimentform. Instead, modifications, equivalents and replacements included inthe disclosed concept and technical scope of this specification may beemployed. While describing exemplary embodiments, if it is determinedthat the specific description regarding a known technology obscures thegist of the disclosure, the specific description is omitted.

The terms such as “first,” “second,” and so on may be used to describe avariety of elements, but the elements should not be limited by theseterms. The terms used herein are solely intended to explain specificexample embodiments, and not to limit the scope of the disclosure.

Singular forms are intended to include plural forms unless the contextclearly indicates otherwise. The terms “include”, “comprise”, “isconfigured to,” etc., of the description are used to indicate that thereare features, numbers, steps, operations, elements, parts or combinationthereof, and they should not exclude the possibilities of combination oraddition of one or more features, numbers, steps, operations, elements,parts or a combination thereof.

The expression at least one of A and B is to be understood as indicatingeither “A” or “B” or “A and B”.

In the disclosure, a ‘module’ or a ‘unit’ performs at least one functionor operation and may be implemented by hardware or software or acombination of the hardware and the software. In addition, a pluralityof ‘modules’ or a plurality of ‘units’ may be integrated into at leastone module and may be at least one processor except for ‘modules’ or‘units’ that should be realized in a specific hardware.

The example embodiments of the disclosure will be described in greaterdetail below in a manner that will be understood by one of ordinaryskill in the art. However, exemplary embodiments may be realized in avariety of different configurations, and not limited to descriptionsprovided herein. Also, well-known functions or constructions are notdescribed in detail since they would obscure the invention withunnecessary detail.

Hereinafter, exemplary embodiments will be described in greater detailwith reference to the accompanying drawings.

FIG. 1 is a view illustrating a peripheral image including a trafficsign to support understanding of the disclosure.

Referring to FIG. 1 , the driving assistance apparatus according to anembodiment of the disclosure may monitor a surrounding environment usinga sensor provided. For example, the driving assistance apparatus mayacquire a peripheral image 10 by capturing a surrounding environment,road conditions in front, or the like using an image capturing part. Theimage capturing part is a component that captures the surroundingenvironment of a vehicle and events occurring on a driving path of thevehicle, and may be referred to as a sensor, a camera, and an imagesensor, but hereinafter, for convenience of description, it will bereferred to as the image capturing part in the disclosure.

The driving assistance apparatus according to an embodiment of thedisclosure may acquire a peripheral image 10 through an image capturingpart, identify a traffic sign 20 area included in the peripheral image10, and provide the identified information to the user. Hereinafter,various embodiments of the disclosure will be described in detail withreference to the drawings.

FIG. 2 is a block diagram indicating a configuration of a drivingassistance apparatus according to an embodiment of the disclosure. Thedriving assistance apparatus 100 of FIG. 2 may be mounted on a vehicleand used. Specifically, the driving assistance apparatus 100 may beimplemented as an electric system of a vehicle or a camera moduleinstalled inside the vehicle. Alternatively, it may be implemented as aroom mirror-integrated module, and may be implemented in a form of aportable device such as a mobile phone, a PDA, or the like that isdetachable to the vehicle. Alternatively, it may be implemented as thevehicle itself.

Referring to FIG. 2 , the driving assistance apparatus 100 includes animage capturing part 110 and a processor 120.

The image capturing part 110 may capture a peripheral image 10 of avehicle. For example, traffic signs, other vehicles, or the like locatedaround the vehicle may be captured through the image capturing part 110.The image capturing part 110 may be implemented with a single camera ora plurality of cameras.

The processor 120 controls overall operation of the driving assistanceapparatus 100.

According to an embodiment, the processor 120 may be implemented as adigital signal processor (DSP), a microprocessor, or a time controller(TCON) that processes digital signals, but is not limited thereto. Theprocessor 120 may include one or more of central processing unit (CPU),microcontroller unit (MCU), micro processing unit (MPU), controller, orcommunication processor (CP), ARM processors, or may be defined as thecorresponding term. In addition, the processor 120 may be implemented ina form of a system on chip (SoC) or large scale integration (LSI) with abuilt-in processing algorithm, or a field programmable gate array(FPGA).

The processor 120 may acquire the peripheral image 10 captured throughthe image capturing part 110. The peripheral image 10 may be aperipheral image of a vehicle in which the driving assistance apparatus100 is mounted. For example, the processor 120 may acquire a peripheralimage 10 including a traffic sign, etc. in real time through the imagecapturing part 110. The traffic signs may be signs displayed in aconsistent and unified manner for traffic safety, and there may be typessuch as caution signs, regulatory signs, instruction signs, auxiliarysigns, road signs, or the like, and shapes such as round, triangle,square, octagon, etc.

The processor 120 may identify a specific color area in the peripheralimage 10 as a first candidate area. The processor 120 may identify thefirst candidate area based on color information included in theperipheral image 10. The candidate area refers to an area that becomes acandidate in which a traffic sign may exist.

Specifically, the processor 120 may identify red (R) color, green (G)color, and blue (B) color values included in the peripheral image 10. Inother words, the processor 120 may identify R, G, and B color valuesincluded in the peripheral image 10 as color information. The colorvalue may refer to a pixel value, and each pixel value of R, G, and Bmay be represented as one of 0 to 255 according to brightness. Forexample, if a pixel value of color R is 255, a pixel value of color Gand color B is 0, the pixel may be represented in red, and if the pixelvalue of color R, color G, and color B is 0, the pixel may berepresented in black.

The processor 120 may identify an area in which the pixel value of the Rcolor is equal to or greater than a predetermined multiple of the pixelvalues of the G color and the B color as a first candidate area. Whenthe pixel value of the R color is larger than the rest, thecorresponding pixel may represent color in R.

Meanwhile, borders of caution signs to inform road users to take safetyactions when there are dangerous substances or road conditions aredangerous, and regulatory signs to notify restrictions that arerestricted or prohibited for safety of road traffic generally use redcolor. Accordingly, the processor 120 may identify an area in which theR color pixel value is higher than the remaining colors in theperipheral image 10, as a candidate area of the traffic sign.

When the first candidate area is identified, the processor 120 mayacquire a binary image including only the first candidate area of theperipheral image 10. The binary image may be an image in which theperipheral image 10 is expressed with only two colors.

A process of acquiring the binary image may be performed using Equation1 below. The binary image acquired through Equation 1 may be representedonly in black color and white color.

$\begin{matrix}{G_{({i,j})} = \{ \begin{matrix}{0,} & {{{{if}\mspace{14mu}{{MAX}( {R,G,B} )}} = R},{R \geq {\sigma_{G} \cdot G}},{{{and}\mspace{14mu} R} \geq {\sigma_{G} \cdot B}}} \\{{{255},}\ } & {Otherwise}\end{matrix} } & \lbrack {{Equation}\mspace{14mu} 1} \rbrack\end{matrix}$

G_((i,j)) is a pixel value on a binary image of (i, j) coordinates, andR is a pixel value of R color, G is a pixel value of G color, B is apixel value of B color, σ_(G) and σ_(B) are predetermined multiplemultiplied by the pixel of the G color and the pixel value of color B,respectively.

The peripheral image 10 may be changed to a binary image having a pixelvalue of 0, and 255 otherwise, when the pixel value of the R color isequal to or greater than the predetermined multiple of the pixel valueof the G color and the B color.

Specifically, the pixel value of the R color is the largest (MAX(R, G,B)=R) among the R, G, and B colors, and the pixel value of the R coloris greater than a value obtained by multiplying each of predeterminedmultiples of σ_(G) and σ_(B) by G pixel value and B pixel value, theprocessor 120 may represent the corresponding pixel in black color(pixel value of 0), and the remaining pixels may be represented in whitecolor (pixel value of 255). In other words, the processor 120 mayacquire a binary image representing an area having a high pixel valueratio of R color in black and the remaining area in white. However, inEquation 1, the area with a high pixel value ratio of the R color isrepresented in black and the remaining area is represented in white, butthis is only an example, and if the first candidate area in theperipheral image 10 is a form that can be distinguished from theremaining area, it may be represented in various ways.

The predetermined multiple may be updated based on at least one ofilluminance measured through an illuminance sensor (not illustrated) andrainfall measured through a rain sensor (not illustrated). In otherwords, the predetermined multiple may be changed according to thesurrounding environmental conditions.

For example, a predetermined multiple when the surrounding environmentof the vehicle is very bright may be a value greater than apredetermined multiple when the surrounding environment is very dark.When the surrounding environment is very bright, since the pixel valueof R color is also measured high, a predetermined multiple may beincreased to discriminate only when the pixel value of R color has alarger difference compared to the G color and the B color. Accordingly,the accuracy in which the processor 120 identifies the R color systemmay be increased. In addition, when the surrounding environment is verydark, the difference between the pixel values of the peripheral image 10is smaller than when the surrounding environment is bright, and thus thepredetermined multiple may be reduced to a relatively small value.

The predetermined multiple may be changed to various values according toan illuminance level, as shown in Table 1. As such, predeterminedmultiples σ_(G) and σ_(B) may be measured through experiments in variousenvironments and stored in a storage unit (not shown) in a form of alookup table.

TABLE 1 Illuminance level σ_(G) σ_(B) Level 1 (darkest) 1 1 Level 2(dark) 1.1 1.1 Level 3 (bright) 1.2 1.2 Level 4 (brightest) 1.3 1.3

The lookup table shown in Table 1 may be used to identify an area wherea R color ratio is higher than that of the other colors in theperipheral image 10, based on illuminance information measured by anilluminance sensor. In Table 1, each illuminance level is classifiedinto darkest, dark, bright, and brightest, and this indicates a relativeilluminance level. This classification of the illuminance level is anexample, and it is possible to accurately distinguish the illuminancelevel according to a threshold value by setting a plurality ofilluminance threshold values.

Meanwhile, a numerical value regarding the predetermined multiple in thetable 1 σ_(G) and σ_(B) is merely an example, and may be changed to adifferent value depending on various environmental conditions such aschange of the type of sensor, or the like. In addition, in Table 1 σ_(G)and σ_(B) are described as the same value, but may have differentvalues.

Meanwhile, the predetermined multiple may be changed based on rainfallmeasured through a rain sensor. For example, a predetermined multipleaccording to the degree of rainfall may be measured and stored in theform of the lookup table.

Meanwhile, information on illuminance and rainfall may be received froman external server (not shown) without measuring illuminance andrainfall through a sensor.

The processor 120 may more accurately identify a traffic sign area byadjusting the predetermined multiple in consideration of the surroundingenvironment of the vehicle.

According to an embodiment of the disclosure, a method of identifying anR color area through a size between pixel values of R, G, and B colorsis a simple computation and may reduce the amount of computation of theprocessor 120.

In addition, it is possible to reduce the amount of computation of theprocessor 120 by identifying candidate areas of traffic signs on theperipheral image 10 and identifying the traffic signs only in thecorresponding candidate areas.

However, depending on setting, an area having a large pixel value of Rcolor may be identified as a candidate area of a traffic sign eventhough it is not a red color but a yellow color.

According to an embodiment of the disclosure, the processor 120 mayidentify an area including a specific shape within the first candidatearea as a second candidate area. The processor 120 may identify thesecond candidate area based on shape information included in the firstcandidate area. Specifically, the processor 120 may identify the secondcandidate area based on the binary image acquired according to the colorinformation. In other words, the processor 120 may, in an area where thepixel value of R color is equal to or greater than the predeterminedmultiple of the pixel values of G color and B color is represented inblack color, and the remaining area is represented in white, identify asecond candidate area based on shape information included the arearepresented in black color.

The processor 120 may identify shape information included in the firstcandidate area by moving a window of a predetermined size on the binaryimage around the first candidate area. However, the processor 120 mayidentify shape information included in the first candidate area bymoving the window of the predetermined size on the binary image over theentire image area.

When reference shape information is identified in the first candidatearea, the processor 120 may identify the first candidate area as asecond candidate area. The reference shape information may beinformation on shape of a sample traffic sign. For example, thereference shape information may include triangular shape informationindicating a caution mark, circular shape information indicating aregulatory mark, square shape information, octagon shape information, orthe like. The reference shape information may be stored in a storageunit when manufacturing the driving assistance apparatus 100 or may bereceived from an external server. The external server may be implementedas a cloud server, but is not limited thereto.

In other words, the processor 120 may compare the shape informationacquired in the first candidate area with the reference shapeinformation stored in the storage unit, and identify the area includingthe acquired shape information as the second candidate area if theacquired shape information matches the reference shape information. Forexample, if circular shape information is identified on the acquiredbinary image, the processor 120 may identify that the circular shapeinformation matches the reference shape information and identify thearea including the circular shape as the second candidate area.Alternatively, if diamond shape information is identified on theacquired binary image, the processor 120 may identify that the diamondshape information does not match the reference shape information, suchthat the processor 120 may not identify the corresponding area as thesecond candidate. In other words, the diamond shape area may beidentified as an area in which no traffic sign exists.

The processor 120 may identify the second candidate area as a pluralityof blocks. Specifically, the processor 120 may identify the secondcandidate area as a plurality of first blocks based on a firstdirection, and may identify the second candidate area as a plurality ofsecond blocks based on a second direction.

For example, the processor 120 may identify the second candidate area asa plurality of M*N-shaped blocks based on a horizontal and verticaldirections.

The processor 120 may acquire pixel information based on pixel valuesincluded in each of the plurality of blocks. Specifically, the processor120 may acquire first pixel information based on pixel values includedin each of the plurality of first blocks, and acquire second pixelinformation based on pixel values included in each of the plurality ofsecond blocks.

The first pixel information may include a difference value between anaverage value of a pixel included in each of the plurality of firstblocks and an average value of pixels included in adjacent first blocks.In addition, the second pixel information may include a difference valuebetween an average value of pixels included in each of the plurality ofsecond blocks and an average value of pixels included in adjacent secondblocks.

For example, it is assumed that the processor 120 identifies the secondcandidate area as 3*3 block.

The first block is composed of a first row, a second row, and a thirdrow block based on the horizontal direction. In addition, the secondblock is composed of a first column, a second column, and a third columnblock based on the vertical direction.

The first pixel information may include an average value of pixelsincluded in each of the first row, second row, and third row blocks, adifference value between the average value of the first row and thesecond row, which are adjacent blocks, and a difference value betweenthe average value of the second row and the third row.

The second pixel information may include an average value of pixelsincluded in each of a first column, a second column, and a third columnblocks, a difference value between average values of the first columnand the second column, which are adjacent blocks, and a difference valuebetween average values of the second column and the third column.

The processor 120 may acquire a feature vector or a matrix valuecorresponding to the acquired first pixel information and the secondpixel information. The feature vector may be a vector listing firstdirection average value information, difference value informationbetween the first direction average values, second direction averagevalue information, and difference value information between the seconddirection average values. A method of acquiring the first pixelinformation and second pixel information described above will bedescribed in detail in FIG. 6 to be described below.

The processor 120 may identify traffic sign area based on the acquiredpixel information. According to an embodiment, the processor 120 mayidentify the traffic sign area based on the first pixel information andthe second pixel information.

According to another embodiment, the processor 120 may apply the firstpixel information and the second pixel information to a training modelto identify the traffic sign area.

The training model may be a model acquired by identifying a plurality ofsample traffic sign images as a plurality of blocks and learning pixelinformation based on pixel values included in each of the plurality ofblocks. The training model may be implemented with a cognitive systemsuch as an artificial neural network or a neuromorphic processor.

For example, the training model may be a model acquired by identifying a“maximum speed limit sign”, which is a sample in a circular shape, as aplurality of M*N blocks and learning pixel information based on pixelvalues included in each of the plurality of blocks. In other words, thetraining model may be a model in which a matrix value corresponding tothe sample traffic sign or feature vector is learned.

The training model may analyze matrix values of input pixel informationand learned pixel information, or a similarity of a feature vector toidentify and output whether an object corresponding to the input pixelinformation is a traffic sign.

When the second candidate area is identified as including the trafficsign by the training model, the processor 120 may identify numbers orsymbols inside the traffic sign through image recognition technology toidentify at least one of a type and indication of the traffic sign. Theimage recognition technology may include pattern matching, opticalcharacter recognition (OCR), or the like. For example, when the secondcandidate area is identified as including the traffic sign by thetraining model, the processor 120 may acquire the number “80” displayedinside the traffic sign through the OCR technology, and identify amaximum speed of the road the vehicle is currently running is 80 Km/h.Alternatively, the type and instruction of traffic signs may beidentified through a training model using a separate artificial neuralnetwork.

The processor 120 may control a display (not shown) to provide at leastone of the identified traffic sign type and instruction. The display maybe provided inside the driving assistance apparatus 100, but is notlimited thereto, and may be implemented as an external display device(not shown). It may be implemented in various forms such as a displayincluded in a navigation provided in a vehicle, a head up display (HUD)displayed on an area of a windshield of a vehicle, a display displayedon an instrument panel area. The processor 120 may transmit informationon at least one of the identified traffic sign type and instruction toan external display device to display the traffic sign type andinstruction content on the external display device.

The processor 120 may control the vehicle based on at least one of theidentified traffic sign type and instruction content. For example, whenthe identified traffic sign is a stop sign, the processor 120 maycontrol the vehicle to stop. Alternatively, when the instruction contentof the identified traffic sign is a speed limit sign with a maximumspeed of 100 Km/h or less, the processor 120 may control the maximumspeed of the vehicle to not exceed 100 Km/h.

The processor 120 may provide feedback notifying at least one of theidentified traffic sign type and instruction content. For example, thefeedback may have a form of vibrating a part of a vehicle such as asteering wheel or outputting a specific sound or voice.

According to another embodiment, the processor 120 may identify atraffic sign by identifying the first candidate area as a plurality ofblocks and applying pixel information acquired based on pixel valuesincluded in each of the plurality of blocks to the training model. Inother words, the processor 20 may identify whether it is a traffic signby omitting a step of identifying the second candidate area based on theshape information, applying the pixel information acquired based on thepixel values included in each of the plurality of blocks to the trainingmodel, and analyzing the shape information in the training model.

In the above, it has been described that the first candidate area isidentified based on color information, the second candidate area isidentified based on the shape information, and the traffic sign isfinally identified within the second candidate area, but the order maybe changed. For example, after identifying the first candidate areabased on the shape information, the second candidate area may beidentified based on the color information. This operation is also justan example, and the order may be arbitrarily changed according to theamount of computation.

FIG. 2 describes based on the driving assistance apparatus 100 includingthe image capturing part 110 and the processor 120, but may beimplemented in a form excluding the image capturing part 110 accordingto an implementation example. In this case, the driving assistanceapparatus may be configured with a memory and/or a processor. Theperipheral image 10 may be acquired through an interface connected to anexternal camera.

FIG. 3 is a block diagram illustrating an example of a detailedconfiguration of a driving assistance apparatus.

According to FIG. 3 , the driving assistance apparatus 100 includes animage capturing part 110, a processor 120, a sensor 130, a storage 140,and a communicator 150. The elements shown FIG. 2 will not be describedin detail.

The processor 120 may include, for example, and without limitation, aCPU, a ROM (or a non-volatile memory) in which a control program forcontrolling the driving assistance apparatus 100 is stored and a RAM (orvolatile memory) used to store data input from outside of the drivingassistance apparatus 100 or used as a storage area corresponding tovarious operations performed in the driving assistance apparatus 100.

When a predetermined event occurs, the processor 120 may execute anoperating system (OS), programs, and various applications stored in thestorage 140. The processor 120 may include a single core, a dual core, atriple core, a quad core, and core of a multiple thereof.

The processor 120 may include an image processor 120-1 and anapplication processor 120-2.

The image processor 120-1 is a component that processes image data. Inparticular, the image processor 120-1 may control an overall operationof identifying a traffic sign area. Specifically, the image processor120-1 may identify a specific color area in a peripheral image capturedby the image capturing part 110 as a first candidate area, identify anarea including a specific shape within the first candidate area is as asecond candidate area, and identify an area of a traffic sign within thesecond candidate area. Since the operation of identifying the trafficsign area has been described above, a detailed description will beomitted.

The application processor 120-2 may drive various applications and is acomponent that performs graphic processing. The application processor120-2 may access the storage 140, and perform booting by using thestored operating system (OS) of the storage 140. Further, theapplication processor 120-2 may perform various operations using variouskinds of programs, content, and data stored in the storage 140. Inaddition, the application processor 120-2 may access web database toperform various operations, and may use sensing information receivedfrom the sensors 130.

The sensor 130 is a component that senses information on the surroundingenvironment of the vehicle. In particular, the sensor 130 may include anilluminance sensor and a rain sensor. The illuminance sensor is acomponent that measures illuminance of the surrounding environment ofthe vehicle, and the rain sensor is a sensor that measures rainfallaround the vehicle.

In addition, the sensor 130 may include a global positioning system(GPS), an inertial measurement unit (IMU), a RADAR unit, a LIDAR unit,and an image sensor. In addition, the sensor 130 may include at leastone of a temperature/humidity sensor, an infrared sensor, an atmosphericpressure sensor, and a proximity sensor, but is not limited thereto, andmay include various types of sensors that detects information on thesurrounding environment of the vehicle, and provides it to the driverand processor 120.

In addition, the sensor 130 may include a motion sensing device capableof sensing the movement of the vehicle. The motion sensing device mayinclude a magnetic sensor, an acceleration sensor, and a gyroscopesensor.

The GPS is a component that detects a geographic location of a vehicle,and the processor 120 may also acquire location information detectedthrough GPS when a peripheral image 10 is acquired.

The IMU may be a combination of sensors configured to detect changes inthe vehicle's position and orientation based on inertial acceleration.For example, the combination of sensors may include accelerometers andgyroscopes.

The RADAR unit may be a sensor configured to detect objects in theenvironment in which the vehicle is located using a wireless signal. Inaddition, the RADAR unit may be configured to detect a speed and/ordirection of objects.

The LIDAR unit may be a sensor configured to detect objects in theenvironment in which the vehicle is located using a laser.

The storage 140 may store various data, programs or applications whichare used to drive and control the driving assistance apparatus 100. Thestorage 140 may store a control program for controlling the drivingassistance apparatus 100 and the processor 120, an application initiallyprovided by a manufacturer or downloaded from an external source,databases, or related data.

In particular, the storage 140 may store information on a referenceshape of a traffic sign. The reference shape information may beinformation on a shape of the traffic sign. For example, the referenceshape information may include triangular shape information indicating acaution sign, circular shape information indicating a regulatory sign,square shape information, octagon shape information, or the like.

The storage unit 140 may be implemented as an internal memory such asROM, RAM, etc. included in the processor 120, or may be implemented as amemory separate from the processor 120. The storage unit 140 may beimplemented in a form such as a non-volatile memory, a volatile memory,a hard disk drive (HDD), or a solid state drive (SSD).

However, according to another embodiment of the disclosure, it ispossible to receive reference shape information from a server (notshown). The server may be implemented as a cloud server, but is notlimited thereto.

The communicator 150 may further perform communication with an externalapparatus (not illustrated). As an example, the communicator 150 maycommunicate with an external server according to a wired/wirelesscommunication method, such as BlueTooth (BT), Wireless Fidelity (WI-FI),Zigbee, Infrared (IR), serial interface, universal serial bus (USB),near field communication (NFC), Vehicle to Everything (V2X), mobilecommunication (Cellular), or the like.

The communicator 150 may receive information on the reference shape of atraffic sign from an external server. In addition, the communicator 150may receive various information related to a current location of avehicle and route from the external server. For example, thecommunicator 150 may receive weather information, news information, roadcondition information, or the like from the external server. The roadcondition information may mean condition information on a path (or road)on which the vehicle is traveling. For example, various information suchas road surface condition information, traffic condition information,traffic accident information, traffic control information, or the likemay be received.

A display (not shown) displays various contents including vehicledriving information. The vehicle driving information may include acurrent speed of the vehicle, a speed limit of a road on which thevehicle is currently running, traffic sign information, or the like. Inparticular, the display may display at least one of a type andinstruction content of the traffic sign identified by a control of theprocessor 120.

The display may be implemented as various forms such as liquid crystaldisplay (LCD), organic light-emitting diode (OLED), liquid crystal onsilicon (LCoS), digital light processing (DLP), quantum dot (QD), microlight-emitting diode (LED) display, etc. In particular, the display maybe implemented in a form of a touch screen forming a layer structurewith a touch pad. In this case, the display may be used as a userinterface (not shown) other than the outputter. The touch screen may beconfigured to detect not only touch input location and area, but also atouch input pressure.

However, the driving assistance apparatus 100 may be implemented in astate that does not include the above-described display.

FIGS. 4A and 4B are views illustrating a process of identifying a firstcandidate area according to an embodiment of the disclosure.

FIG. 4A is a peripheral image 410 of a vehicle obtained through theimage capturing part 110. The peripheral image 410 may includesurrounding environments such as a path on which the vehicle is running,an external vehicle, a traffic sign, or the like.

The driving assistance apparatus 100 may identify the first candidatearea based on color information included in the peripheral image 410.Specifically, the driving assistance apparatus 100 may identify an areain which the pixel value of the R color included in the peripheral image410 is equal to or greater than a predetermined multiple of the pixelvalues of the G color and the B color as the first candidate area.

According to FIG. 4A, the pixel value of R color of an areacorresponding to an octagonal object 420 and a taillight 430 of anexternal vehicle may be identified as an area that is equal to orgreater than the predetermined multiple of the pixel values of the Gcolor and the B color, and identified as a first candidate area.

In this case, the driving assistance apparatus 100 may obtain a binaryimage 440 including only the first candidate area.

FIG. 4B is a binary image 440 in which only the first candidate area isrepresented in black color and the remaining areas are represented inwhite color.

Accordingly, only a rim of the octagonal object 420 and the taillight430 of the external vehicle may be displayed on the binary image 440.

However, in the binary image 440 according to FIG. 4B, the firstcandidate area is represented in black and the remaining area isrepresented in white, but this is only an example, and if the firstcandidate area in the peripheral image 410 is a form that can bedistinguished from the remaining areas, it may be represented in variousways.

FIGS. 5A and 5B are views illustrating a process of identifying a secondcandidate area according to an embodiment of the disclosure.

FIG. 5A is a binary image 510 including only a first candidate area.

The driving assistance apparatus 100 may identify a second candidatearea based on shape information included in the first candidate area.Specifically, when the reference shape information is identified in thefirst candidate area based on the binary image 510 acquired according tothe color information, the driving assistance apparatus 100 may identifythe identified first candidate area as a second candidate area.

According to FIG. 5A, the shape of an octagonal object 520 included inthe binary image 510 is an octagonal shape. Since the octagonal shape isa shape stored in the reference shape information, the drivingassistance apparatus 100 may identify an area including the octagonalobject 520 as a second candidate area.

In addition, since the shape of the taillight 530 of the externalvehicle included in the binary image 510 is an elliptical shape and isnot a shape stored in the reference shape information, the drivingassistance apparatus 100 may exclude an area including the taillight 530of the external vehicle from the candidate area.

FIG. 5B may be an image including only the second candidate area 550.Only the octagonal object 520 is identified as the second candidate area550 based on the shape information.

The driving assistance apparatus 100 may identify the second candidatearea 550 as a plurality of blocks, and apply pixel information obtainedbased on pixel values included in each of the plurality of blocks to atraining model to identify whether an object included in the secondcandidate area 550 is a traffic sign. This will be described in detailin FIG. 6 .

FIGS. 6A and 6B are views illustrating a process of acquiring pixelinformation by identifying the second candidate area as a plurality ofblocks according to an embodiment of the disclosure.

FIG. 6A is a view illustrating a process of acquiring pixel informationin a second candidate area 610 identified as a plurality of M*N-shapedblocks.

The driving assistance apparatus 100 may identify the second candidatearea 610 based on color information and shape information in theperipheral image, and may identify the second candidate area 610 as theplurality of M*N-shaped blocks. For example, the driving assistanceapparatus 100 may identify the second candidate area 610 as a pluralityof M first blocks based on a horizontal direction, and identify thesecond candidate area 610 as a plurality of N second blocks based on avertical direction.

The driving assistance apparatus 100 may obtain first pixel information620 and 625 based on pixel values included in each of the M firstblocks, and obtain second pixel information 630 and 635 based on pixelvalues included in each of the N second blocks.

The first pixel information 620 and 625 may include a difference value624 between an average value 620 of pixels included in each of theplurality of first blocks and an average value of the pixels included inadjacent first blocks. In addition, the second pixel information 630 and635 may include a difference value 635 between an average value 630 ofpixels included in each of the plurality of second blocks and an averagevalue of pixels included in adjacent second blocks. Also, the firstpixel information and second pixel information 620 to 635 may beobtained in a form of a matrix value or a vector shape. Hereinafter, forconvenience of description, the average value 620 of the pixels includedin each of the plurality of first blocks is referred to as a matrix A₁,and the difference value 625 between the average values of pixelsincluded in the adjacent first blocks is referred to as a matrix D₁, theaverage value 630 of pixels included in each of the plurality of secondblocks is referred to as matrix A₂, and the difference value 635 betweenthe average values of pixels included in adjacent second blocks isreferred to as matrix D₂.

These matrices may be calculated through the following equation.

$\begin{matrix}{{{A_{1}(k)} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}A_{({k,i})}}}},{1 \leq k \leq M}} & \lbrack {{Equation}\mspace{20mu} 2} \rbrack \\{{{A_{2}(r)} = {\frac{1}{M}{\sum\limits_{i = 1}^{M}A_{({i,r})}}}},{1 \leq r \leq N}} & \lbrack {{Equation}\mspace{14mu} 3} \rbrack\end{matrix}$

Here, the matrix A₁ is a matrix related to the average values of pixelsincluded in each of the plurality of first blocks, the matrix A₂ is amatrix related to the average values of pixels included in each of theplurality of second blocks, and M is the number of columns, N is thenumber of rows, and A is a pixel value.

Matrix information 620 and 630 about the average value may includespatial information on the shape of the traffic signs, but it may not beeasy to distinguish between similar shapes represented by triangles orsquares. Therefore, by using not only the information on average values,but also information on matrixes 625 and 635 about difference betweenadjacent average values, it may more accurately calculate the pixelinformation for the traffic sign to be identified.D ₁(t)=|A ₁(t+1)−A ₁(t)|,1≤M−1  [Equation 4]D ₂(S)=|A ₂(s+1)−A ₂(s)|,1≤s≤N−1  [Equation 5]

Here, the matrix D₁ is a matrix regarding difference values betweenaverage values of pixels included in adjacent first blocks, and thematrix D₂ is a matrix regarding difference values between average valuesof pixels included in adjacent second blocks.

An A1 matrix and a D1 matrix in an M*1 form, and an A2 matrix and a D2matrix in a 1*N form may be calculated through Equations 2 to 5described above.

The calculated A1, D1, A2, and D2 matrices may be applied to a trainingmodel as pixel information, so that whether a traffic sign is includedin the second candidate area may be identified.

It may be changed to a 1*M form through a transpose matrix of A1 matrixand D1 matrix in the M*1 form, such that the A1, A2, D1, and D2 matricesmay all be applied to a training model in a unified form having only onerow. Alternatively, it may be changed to an N*1 form through a transposematrix of A2 matrix and D2 matrix in the 1*N form, such that the A1, A2,D1 and D2 matrices may all be applied to the training model in a unifiedform having only one column.

FIG. 6B is a view illustrating an example of acquiring pixel informationby identifying a second candidate area as a plurality of blocks.

The driving assistance apparatus 100 may acquire pixel information byidentifying the second candidate area 610 as a plurality of M*N-shapedblocks from an acquired peripheral image 640.

Referring to FIG. 6B, the driving assistance apparatus 100 may computean average value of each pixel of N blocks in row 1 as 83, an averagevalue of each pixel of N blocks in row 2 as 66, and an average value ofeach pixel of N blocks in row M as 47. In addition, the drivingassistance apparatus 100 may compute 17, which is a difference valuebetween the average values of the first row and the second row, whichare adjacent blocks, and 15, which is a difference between the averagevalues of the second row and the third row, which are adjacent blocks.

In addition, the driving assistance apparatus 100 may compute an averagevalue of each pixel of M blocks in column 1 as 86, an average value ofeach pixel of M blocks in column 2 as 67, and an average value of eachpixel of M blocks in column N as 45. In addition, the driving assistanceapparatus 100 may compute 19, which is a difference value between theaverage values of the first column and the second column, which areadjacent blocks, and 13, which is a difference between the averagevalues of the second column and the third column, which are adjacentblocks.

The driving assistance apparatus 100 may compute an average value of thepixels included in each of the first blocks 620, a difference valuebetween the average values of the pixels included in the adjacent firstblock 625, average value of pixels included in each of the plurality ofsecond blocks 630, and a difference value between the average values ofpixels included in the adjacent second blocks 635.

Then, the driving assistance apparatus 100 may apply the pixelinformation 620 to 635 acquired based on the pixel values included ineach of the plurality of blocks to the training model, such that theapparatus may identify whether an object included in the secondcandidate area 610 is a traffic sign.

When the second candidate area 610 is identified as including a trafficsign by the training model, the driving assistance apparatus 100 mayidentify numbers or symbols inside the traffic sign for the secondcandidate area through image recognition technology to identify at leastone of a type and an instruction content of the sign. For example, thedriving assistance apparatus 100 may obtain the number “60” displayedinside the traffic sign through OCR technology, and identify that amaximum speed of the road on which the vehicle is currently driving is60 Km/h.

FIGS. 7A and 7B are views illustrating an operation of providinginformation on an identified traffic sign area to a user according to anembodiment of the disclosure.

According to FIG. 7A, the driving assistance apparatus 100 may display aperipheral image acquired by an image capturing part through a display710 provided in a dashboard, a navigation system, or the like.

In particular, when the second candidate area is identified as includinga traffic sign 720 by the training model, the driving assistanceapparatus 100 may provide at least one of the type and instructioncontent of the traffic sign identified through OCR technology throughthe 710.

For example, as shown in FIG. 7A, the driving assistance apparatus 100may acquire the number “60” displayed inside the traffic sign 720 toidentify that the maximum speed of the road the vehicle is currentlydriving is 60 Km/h. The driving assistance apparatus 100 may provideinformation that the speed limit of the road that the vehicle iscurrently driving is 60 Km/h to the user through the display 710. Inaddition, the driving assistance apparatus 100 may automatically controlthe vehicle such that the maximum speed of the vehicle does not exceed60 Km/h.

Referring to FIG. 7B, the driving assistance apparatus 100 may display aperipheral image acquired by the image capturing part through a HUDdisplay 730. In addition, the driving assistance apparatus 100 maydisplay information on the traffic sign 720 through the HUD display 730.

For example, the driving assistance apparatus 100 may provideinformation that the speed limit of the road that the vehicle is drivingis 60 Km/h to the user through the HUD display 730. In addition, thedriving assistance apparatus 100 may automatically control the vehiclesuch that the maximum speed of the vehicle does not exceed 60 Km/h.

FIG. 8 is a flowchart illustrating an image processing method accordingto an embodiment of the disclosure.

The driving assistance apparatus may acquire a peripheral image of avehicle through the image capturing part (S810).

The driving assistance apparatus may identify a first candidate areabased on color information included in the peripheral image (S820).

Specifically, the driving assistance apparatus may identify red (R)color, green (G) color, and blue (B) color values included in theperipheral image, and identify an area where a pixel value of R color isequal to or greater than a predetermined multiple of pixel values of Gand B color as the first candidate area.

The predetermined multiple may be updated based on at least one ofilluminance and rainfall of surrounding environment.

The driving assistance apparatus may identify a second candidate areabased on shape information included in the first candidate area (S830).

Specifically, when the first candidate area is identified, the drivingassistance apparatus may acquire a binary image including only the firstcandidate area among the peripheral images, and identify the secondcandidate area based on the acquired binary image. have. When referenceshape information of a traffic sign pre-stored in the first candidatearea is identified, the driving assistance apparatus may identify thefirst candidate area as the second candidate area.

The driving assistance apparatus may identify the second candidate areaas a plurality of blocks, and may identify a traffic sign area based onpixel values included in each of the plurality of blocks (S840).

Specifically, the driving assistance apparatus may identify the secondcandidate area as a plurality of first blocks based on a firstdirection, acquire first pixel information based on pixel valuesincluded in each of the plurality of first blocks, identify the secondcandidate area as a plurality of second blocks based on a seconddirection, and acquire second pixel information based on pixel valuesincluded in each of the plurality of second blocks. Then, the drivingassistance apparatus may identify the traffic sign area based on thefirst pixel information and the second pixel information.

The first pixel information may include a difference value between anaverage value of a pixel included in each of the plurality of firstblocks and an average value of a pixel included in adjacent firstblocks. In addition, the second pixel information may include adifference value between an average value of pixels included in each ofthe plurality of second blocks and an average value of pixels includedin adjacent second blocks.

The driving assistance apparatus may identify a traffic sign area byapplying the first pixel information and the second pixel information toa training model. The training model may be a model acquired byidentifying a plurality of sample traffic sign images as a plurality ofblocks and learning pixel information based on pixel values included ineach of the plurality of blocks.

The driving assistance apparatus may provide at least one of a type andan instruction content of the identified traffic sign.

In addition, the driving assistance apparatus may be automaticallycontrolled based on at least one of the type and instruction content ofthe traffic sign.

Since detailed operations of each step have been described above,detailed descriptions will be omitted.

Meanwhile, at least some of the above-described methods according tovarious embodiments of the disclosure may be installed in an existingdriving assistance apparatus, and may be implemented in a form of anapplication, which is software directly used by the user on an OS. Theapplication may be provided in a form of an icon interface on a displayscreen of the vehicle.

Further, the methods according to the above-described exampleembodiments may be realized by upgrading the software or hardware of theexisting electronic apparatus.

Various exemplary embodiments described above may be embodied in arecording medium that may be read by a computer or a similar apparatusto the computer by using software, hardware, or a combination thereof.In some cases, the embodiments described herein may be implemented by aprocessor itself. In a software configuration, various embodimentsdescribed in the specification such as a procedure and a function may beembodied as separate software modules. The software modules mayrespectively perform one or more functions and operations described inthe present specification.

Methods of controlling a display apparatus according to variousexemplary embodiments may be stored on a non-transitory readable medium.

The non-transitory computer readable recording medium refers to a mediumthat stores data and that can be read by devices. For example, thenon-transitory computer-readable medium may be CD, DVD, a hard disc,Blu-ray disc, USB, a memory card, ROM, or the like.

The foregoing exemplary embodiments and advantages are merely exemplaryand are not to be construed as limiting the disclosure. The presentteaching can be readily applied to other types of apparatuses. Also, thedescription of the exemplary embodiments is intended to be illustrative,and not to limit the scope of the claims, and many alternatives,modifications, and variations will be apparent to those skilled in theart.

What is claimed is:
 1. A driving assistance apparatus comprising: animage sensor; and a processor configured to identify a specific colorarea in a peripheral image captured by the image sensor as a firstcandidate area, identify an area including a specific shape in the firstcandidate area as a second candidate area, and identify an area of atraffic sign in the second candidate area, wherein the processor isconfigured to: identify the second candidate area as a plurality offirst blocks based on a first direction, and acquire first pixelinformation based on a pixel value included in each of the plurality offirst blocks, identify the second candidate area as a plurality ofsecond blocks based on a second direction, and acquire second pixelinformation based on a pixel value included in each of the plurality ofsecond blocks, and identify an area of the traffic sign based on thefirst pixel information and the second pixel information, wherein thefirst pixel information includes a difference value between an averagevalue of pixels included in each of the plurality of first blocks, andan average value of pixels included in adjacent first blocks, andwherein the second pixel information includes a difference value betweenan average value of pixels included in each of the plurality of secondblocks, and an average value of pixels included in adjacent secondblocks.
 2. The apparatus of claim 1, wherein the processor is configuredto apply the first pixel information and the second pixel information toa training model to identify the area of the traffic sign, and whereinthe training model is configured to identify a plurality of sampletraffic sign images as a plurality of blocks, and learn and obtain pixelinformation based on pixel values included in each of the plurality ofblocks.
 3. A driving assistance apparatus comprising: an image sensor;at least one among an illuminance sensor and a rain sensor; and aprocessor configured to identify a specific color area in a peripheralimage captured by the image sensor as a first candidate area, identifyan area including a specific shape in the first candidate area as asecond candidate area, and identify an area of a traffic sign in thesecond candidate area, wherein the processor is configured to: identifyred (R) color, green (G) color, and blue (B) color included in theperipheral image, and identify an area in which a pixel value of the Rcolor is equal to or greater than a predetermined multiple of pixelvalues of the G color and the B color as the first candidate area, andwherein the multiple is updated based on at least one of illuminancemeasured by the illuminance sensor or rainfall measured by the rainsensor.
 4. The apparatus of claim 1, wherein the processor is configuredto, based on the first candidate area being identified, acquire a binaryimage including only the first candidate area among the peripheralimage, and identify the second candidate area based on the acquiredbinary image.
 5. The apparatus of claim 1, further comprising: a storageconfigured to store reference shape information of a traffic sign, basedon the reference shape information being identified in the firstcandidate area, identifying the first candidate area as the secondcandidate area.
 6. The apparatus of claim 1, further comprising: adisplay, wherein the processor is configured to control the display toidentify and provide at least one of a type or an instruction content ofthe traffic sign in the identified traffic sign area, and based on atleast one of the type or the instruction content of the traffic sign,control the vehicle.
 7. A method of image processing comprising:acquiring a peripheral image of a vehicle through an image sensor;identifying a first candidate area based on color information includedin the acquired peripheral image; identifying a second candidate areabased on shape information included in the first candidate area; andidentifying the second candidate area as a plurality of blocks, andidentifying an area of traffic sign based on pixel values included ineach of the plurality of blocks, wherein the identifying the trafficsign area comprises: identifying the second candidate area as aplurality of first blocks based on a first direction, and acquiringfirst pixel information based on a pixel value included in each of theplurality of first blocks, identifying the second candidate area as aplurality of second blocks based on a second direction, and acquiringsecond pixel information based on a pixel value included in each of theplurality of second blocks, and identifying an area of the traffic signbased on the first pixel information and the second pixel information,wherein the first pixel information includes a difference value betweenan average value of pixels included in each of the plurality of firstblocks, and an average value of pixels included in adjacent firstblocks, and wherein the second pixel information includes a differencevalue between an average value of pixels included in each of theplurality of second blocks, and an average value of pixels included inadjacent second blocks.
 8. The method of claim 7, wherein theidentifying the traffic sign area comprises applying the first pixelinformation and the second pixel information to a training model toidentify the area of the traffic sign, and wherein the training model isconfigured to identify a plurality of sample traffic sign images as aplurality of blocks, and learn and obtain pixel information based onpixel values included in each of the plurality of blocks.
 9. The methodof claim 7, wherein the identifying the first candidate area comprises:identifying red (R) color, green (G) color, and blue (B) color includedin the peripheral image, and identifying an area in which a pixel valueof the R color is equal to or greater than a predetermined multiple ofpixel values of the G color and the B color as the first candidate area.10. The method of claim 9, wherein the multiple is updated based on atleast one of illuminance or rainfall.